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Abstract 

After describing implicit theory that links school accountability to school 
performance, this article explains how three main issues keep the theory from working in 
practice. The issues involve a) implementation controversies dealing with standards, 
incentives and constituencies; b) insufficient efforts to build organizational capacity and c) 
failure to recognize the importance of internal school accountability. A study of 24 
restructuring schools showed that strong accountability was rare; organizational capacity was 
not related to accountability; schools with strong external accountability tended to have low 
organizational capacity; and strong internal accountability tended to reinforce a school’s 
organizational capacity. Implications for accountability policy are discussed. 
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Overview 



Agencies external to schools, especially local school districts and states, have legal 
and political responsibility for the quality of education in the United States. The extent of 
responsibility at the different levels of government varies and is subject to dispute, but there 
is wide agreement that delivering education is ultimately a responsibility not simply of each 
individual school, but of the broader polity which funds and grants schools the authority to 
operate. 

Spurred by reports of high drop-out rates among smdents from disadvantaged 
backgrounds and overall low smdent achievement, districts, states and other agents external 
to schools, especially business firms and organizations of citizens and parents, have become 
increasingly concerned to establish policies that elevate school performance. A prominent 
strategy has been to try to strengthen school accountability. According to Kirst (1990), 40 
states increased their accountability mechanisms during the 1980’s. Will increased 
accountability of schools to external agents improve student performance throughout the 
United States? 

As part of a smdy of school restrucmring, we examined the namre and extent of 
accountability in 24 "restrucmring" elementary, middle and high schools, located in 16 states 
and 22 districts. We were interested in the forms of accountability apparent in these schools, 
whether some schools experienced more comprehensive accountability demands than others, 
and what effects "strong" versus "weak" accountability seemed to have on teaching and 
learning in the schools. The design of the smdy did not allow us to specify direct 
connections between school accountability and smdent achievement. But analysis of how 



accountability worked in the schools revealed three main problems with the proposition that 
strong external accountability will enhance school performance. 

The first problem is the difficulty of implementing comprehensive external 
accountability across U.S. schools. The second is the recognition that even if external 
accountability could be implemented, this would not guarantee high performance of schools 
that began with low organizational capacity. The third is that strong accountability can occur 
internally within a school community; it need not be prompted by demands from external 
agencies.' This article explains these points and discusses their implications for district, 
state or federal policy aimed at enhancing school accountability. To set the stage we first 
define accountability and its assumed connection to organizational performance. 

Accountability and Organizational Performance: the "Theory" 

Historically, the concept of accountability has reflected a relationship between a 
steward or provider of a good or service and a patron or agent with the power to reward, 
punish, or replace the provider (Kirst, 1990). A variety of criteria could be applied to the 
provider’s behavior and performance. For schools in the contemporary setting, 
accountability can be defined as the process by which school districts and states, or other 
constituents such as parents, attempt to ensure that schools and schools systems meet their 
goals. 

Accordingly, a complete school accountability system should include at least four 

parts: 

1. Information about the organization’s performance (e.g. test scores). 
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2. Standards for judging the quality or degree of success of organizational 
performance (e.g. school mean achievement score to be in the top half of the 
distribution for schools with comparable demographic characteristics). 

3. Significant consequences to the organization (i.e. rewards and sanctions) for its 
success and failure (e.g. bonuses to teachers in the school for the years when 
the school meets or exceeds the standard). 

4. An agent or constituency that receives information on organizational 
performance. Judges the extent to which standards have been met, and 
distributes rewards and sanctions (e.g. the state department of instruction).^ 

How would these four parts of an accountability system enhance school performance? 
The assumption is that teachers will try harder and become more effective in meeting goals 
for student performance when the goals are clear, when information on the degree of success 
is available and when there are real incentives to meet the goals. This theory offers a 
compelling explanation for organizational performance in commercial enterprises where 
customers and clients exercise careful surveillance of the quality of output. High customer 
satisfaction leads to increased demand for the product or service and higher profits which the 
producer will act to maximize. Customer dissatisfaction leads to lower demand and 
economic loss. Assuming that business organizations will behave in ways to continue their 
existence and to maximize the economic rewards, the accountability system is a major 
stimulus for the organization’s workers to produce high quality goods and services. 

Imagine two schools - one with none of these four components, another with all of 
them. School A collects virtually no systematic information on student performance, except 



for teachers’ grades. Within individual courses, the staff administers subject matter tests, but 
they have not used consistent standards to evaluate student performance across subjects or 
over time. Neither the district, state or parents have pressed the school to demonstrate how 
successful it is, and the school never receives any rewards or sanctions related to 
performance. In contrast, school B gathers common information on student achievement each 
year in each of the four main academic subjects by administering performance exams 
developed in a new state assessment scheme. Teachers score the exams according to a set of 
standards that distinguishes among proficient, satisfactory and inadequate student 
performance. The school’s teachers receive bonuses from the district each year that it 
increases the proportion of students who reach the proficient level on all tests. According to 
the theory, if we were to compare academic performance at the two schools - for example, 
by using school B’s tests and standards - school B would show higher performance. But if 
school B’s full accountability system were implemented at school A, the theory suggests that 
school A’s performance would improve over time. 

One way of testing the theory would be to measure the extent that all four 
components of accountability are present for a school and to examine, in a large number of 
schools, the extent to which strength of accountability is associated with a common measure 
of student achievement, independent of other factors that affect student achievement. Due to 
a small sample of schools, diversity among school goals, and lack of comprehensive 
measures of student performance across schools, the present study did not permit such a test. 
But we did examine, in an exploratory fashion, the workings of the four components of 
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accountability in 24 restructuring schools to inquire about the promise of the theory that 
stronger external accountability will elevate school performance. 

Applying the Theory to Schools: A More Complex Picture 
As indicated above, we discovered three main unresolved issues in the theory 
(implementation, organizational capacity, and internal accountability), which we will describe 
in more detail here. This discussion is based on logical analysis of the theory of 
accountability and organizational performance, prior research on school change, and on 
initial examination of experiences among the 24 schools studied. To test these initial 
observations more carefully, we conducted a closer empirical analysis of the school data 
according to the design described later. But before presenting the more specific findings, we 
explain the general observations that launched the subsequent investigation. 

Implementation Issues: Standards. Incentives. Constituencies 

Virtually all of the schools we studied had implemented the first requirement for a 
strong accountability system: they collected information about student performance and 
disseminated it to external agencies, such as the district and state, and to parents. Only 
rarely, however, was this information evaluated according to clear standards for school 
success specified by external agencies, and generally there were no significant material 
incentives for schools to succeed. 

There is widespread agreement in the U.S. that schools should be held more 
accountable to standards for student performance (Johnson & Immerwahr, 1994; Johnson et 
al,, 1995). At the same time, however, controversy persists on how to implement standards 
and what the specific standards should be. At the high school level, for example, there is 



often a perceived conflict between reducing the dropout rate or increasing the graduation rate 
on the one hand, and insisting upon high standards of academic performance as criteria for 
retention in school and graduation. Even when discussion is limited to standards for 
academic achievement, the debate on national standards has revealed hotly contested 
conceptions of desired student outcomes (Apple, 1996). For example, one persisting position 
emphasizes student absorption of knowledge as traditionally organized, and another 
emphasizes student construction of meaning or "teaching for understanding" (Newmann, 
1993). 

Even with general agreement on a traditional or constructivist orientation toward 
teaching and learning, dispute continues over whether school performance should be judged 
according to individual student improvement or on absolute performance standards, whether 
it should be Judged relative to the social background of the student population, and whether 
there should be different standards for students in different programs, such as bilingual, 
special education, college preparatory, or technical- vocational programs. In short, arriving 
at clear standards for school performance involves a thicket of professional and political 
issues that neither districts, states, nor professional organizations have been able to solve.'* 

Linking significant incentives or sanctions to school success and failure poses another 
significant implementation issue, even if standards for success were in place. Strong 
financial incentives, especially teacher bonuses for high school performance, might well 
stimulate greater effort. But without major reallocation of education spending, this strategy 
has little prospect for success, given the public’s reluctance to spend more on education.^ 
Apart from direct financial rewards, it has been argued (e.g. Maeroff, 1988) that high quality 
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professional working conditions, especially more time to plan, to work with mentors and 
participate in professional development, would provide important incentives for teachers to 
perform at higher levels. These conditions can offer powerful rewards, but they are also 
expensive, and if they are critical to school success, should be available in all schools, not 
allocated only to the more successful. 

Consistent with findings by Wohlstetter, Smyer & Mohrman (1994), we found that 
neither restructuring schools nor their teachers received significant material consequences 
based on the performance of their students. Some states and districts have compensated 
teachers or schools to reward school performance and teachers’ demonstration of skills and 
participation in professional development, but these are rare (Kelly & Odden, 1995). Some 
have implemented sanctions for exceptionally poor performance by closing schools or taking 
them over. Voucher systems that fund schools primarily through voluntary subscription by 
parents would, presumably, attach significant consequences to school performance, because 
the school’s very existence would depend upon demonstrating satisfactory performance to 
parents. Some evidence indicates, however, that parents’ choices of schools are not based 
primarily on the schools’ records of high academic performance (Rubenstein, Hammer & 
Adelman, 1992). 

Consideration of the standards and incentives issues raises a third problem in 
implementing strong external accountability; to what external constituency (ies) should schools 
be accountable? Schools are bombarded by demands from different external groups with 
political-legal authority to influence the schools: the local school board and district 
administration, state and federal agencies, and parents. Demands from these groups (and 
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from other interests such as business) vary considerably, and as a whole they fail to act in 
concert to define standards and incentives that could enhance accountability for student or 
school performance (Levin, 1974, Fuhrman, Elmore & Massed, 1993). The challenge to 
coordinate and focus expectations from external groups has been clearly made in the 
argument for "systemic reform" (Smith & O’Day, 1991). To implement a strong 
accountability system will require resolving the political problem of deciding how to allocate 
authority for standard setting and incentives among competing constituencies. 

Organizational Capacity 

We have raised problems of implementing a strong external accountability system, but 
assume for a moment that these difficulties could be overcome and that a strong 
accountability system could be implemented. Would this necessarily enhance school 
performance? Advocates for strong external accountability, as well as critics, recognize that 
the formal components of accountability aren’t enough. First, the standards themselves must 
call for more ambitious, high quality intellectual work for all students. Stricter 
accountability to deliver mediocre curriculum, or to expect more challenging academic work 
only from economically privileged students would be no advance. Second, even if external 
authorities provided higher quality standards and inducements, many schools would lack the 
capacity to meet them. To meet higher standards’, major advances will be needed in the 
quality of technical resources (such as curriculum and assessment materials, laboratory 
equipment, library and computing facilities), in professional development for staff, and in 
finding ways to balance strong external accountability with significant autonomy for schools 
to craft programs that respond to their unique social contexts.* 
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Our study of restructuring schools confirmed the importance of these concerns, and 
we think they can be viewed most usefully as interrelated dimensions of a more fundamental 
concept - school organizational capacity. Concern for building capacity has entered reform 
discourse (Darling -Hammond, 1993; David, 1994; Corcoran & Goertz, 1995; McDonnell & 
Elmore, 1987; O’Day, Goertz, & Floden, 1995), but without unanimity on a specific 
definition.’ Proposed ingredients of organizational capacity include teachers’ professional 
knowledge, effective leadership, availability of technical and financial resources, and 
organizational autonomy to act according to demands of the local context. To the extent that 
these factors reach high levels within a school, one would expect an increase in a school’s 
capacity to deliver high quality instruction which, in turn, ought to produce high quality 
student achievement. 

This formulation makes sense, but, in our view, needs to go one step further. The 
critical defining feature of organizational capacity is the degree to which the human, 
technical and social resources of an organization are organized into an effective collective 
enterprise. The separate items above make critical contributions to organizational capacity, 
but only when they are organized and coordinated to advance the organization’s goals. For 
schools, the key to effective coordination and organization of these resources is shared 
commitment and collaboration among staff to achieve a clear purpose for student learning. 
This shared commitment and collaborative activity is what harnesses otherwise disparate 
technical and human resources into an effective collective enterprise.* 

Our study measured school organizational capacity along three dimensions: teacher 
knowledge and skill, school autonomy to act, and shared commitment and collaboration 



toward a clear purpose for student learning. In many schools, staff disagreed about 
fundamental standards for student success and had great difficulty working together toward a 
common school purpose. Neither the promulgation of external standards for school 
performance, new curriculum and assessment practices, additional staff development, 
provision of advanced technology, or site-based management necessarily generated consensus 
and collaboration for a common mission. Some schools were far more successful than others 
on this dimension, but achieving unity and effective collaboration within schools seemed to 
be a daunting challenge, independent of the extent of external accountability, technical 
resources, professional development, or school autonomy.^ 

Although we define organizational capacity as a property internal to schools, agents 
external to schools can conceivably help to build organizational capacity through provision of 
technical resources (including formulation of high standards for curriculum and 
performance), professional development, and deregulation (Newmann & Wehlage, 1995; 
Wehlage, Osthoff & Porter, 1996). Strong external accountability systems might also help to 
generate or reinforce school consensus and staff collaboration focused on clear goals for 
student learning, but as we shall see, major problems arise in aligning external standards and 
inducements with the internal cultures of schools that define their operating educational 
missions. 

Internal Accountability 

As indicated earlier, the concept of accountability implies strong influence by a 
patron, client or other agent external to a service provider. The agent presumably sets 
standards for performance and distributes incentives and sanctions. In some schools, 
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however, we found that essential components of accountability were generated largely within 
a school staff. Staff identified clear standards for student performance, collected information 
to inform themselves about their levels of success, and exerted strong peer pressure within 
the faculty to meet the goals. In some schools, strong internal accountability was 
accompanied by compatible external accountability, but in others, internal accountability 
existed without, or even in opposition to, external accountability requirements. These 
internally generated accountability systems constituted a major source of cohesion within the 
school. Thus, internal accountability can be seen both as a building block of organizational 
capacity, but also as a result or product of high organizational capacity. 

To summarize, the difficulties of implementing a common set of clear standards and 
powerful inducements across a wide range of schools, the nature and importance of 
organizational capacity to high performance by schools, and the possibility of internally 
generated school accountability present a more complex picture of the connection between 
accountability and school performance. This formulation seems to suggest at least two 
possible implications relevant to the implementation of strong external accountability systems. 
First, to the extent that external accountability is difficult to implement because of issues 
raised above, school accountability might still be achieved through internal mechanisms. 
Second, to the extent that strong external accountability can be implemented, if it 
promulgates standards and incentives hostile to a school’s internal accountability system, it 
may undermine school organizational capacity and thereby defeat its intended purpose. 

Ideally, efforts to increase external accountability should be closely coupled with those that 
try to enhance school organizational capacity. These implications will be clarified as we 



elaborate upon more specific empirical findings on the nature and extent of accountability 
and organizational capacity in 24 restructuring schools. 

Accountability and Organizational Capacity in Restructuring Schools 
Study Design 

This analysis draws on data from the five-year study of school restructuring in the 
USA by the Center on Organization and Restructuring of Schools, University of Wisconsin- 
Madison. Twenty-four schools participated in the study and each met a number of criteria 
for restructuring. Schools were equally divided among elementary, middle and high schools, 
and reflected a broad spectrum of location, size, and student body compositions. Here we 
examine evidence from 20 of the schools. 

The study involved two week-long visits by Center research teams to each school 
during one year, in which they observed classes and meetings, and interviewed teachers, 
administrators, and others active in restructuring. Survey data of all teachers and of students 
in the target grade (5th for elementary, 8th for middle, 10th for high schools), as well as 
school demographic and achievement data, were also obtained. Research teams prepared 
extensive school reports that served as the database for qualitative evidence for each school. 
Two members of each team then coded key aspects of structural and programmatic features 
of their school. School reports, upon which the coding was based, addressed the defining 
components of each school’s accountability system. “ 

We began by asking, what does accountability look like across the sample of 20 
restructuring schools? Based upon the coding of reports and consistent with the framework 



presented above, we determined whether or not a school’s operating accountability system 
included the following: 

• Provisions for information on student performance, explicit standards for 
student performance, and consequences to the school or teachers for student 
success or failure. 

• Information, standards, and/or consequences that were required by an external 
agent (district and/or state), developed by the school itself, or both. 

• An external agent that required something specific beyond mandatory 
standardized testing as part of the school’s accountability system or that 
required the school to develop its own accountability system. 

The appendix (part I) presents a partial list of items from the school codings employed in this 
analysis. 

According to the coding, the district or state required information on performance 
from all schools via standardized testing, but typically failed to specify any standards or 
consequences. For many schools, this information seemed to serve symbolic purposes only. 
At the same time, a number of schools exhibited information, standards, and consequences 
independent of any external requirements. This finding suggested that some schools 
maintained internal accountability systems, and led us to review qualitative data from school 
reports to learn more about the significance of internal vs external agents in establishing 
school accountability. 

To address the concern that school accountability systems need to be stronger and 
more rigorous, we sought to group schools by the relative strength of their existing 
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accountability system. If a school had all components of information, standards, and 
consequences (either externally-required, internally-generated, or a combination), it was 
considered to have a strong accountability system. At the other extreme, if a school had no 
information beyond standardized testing, no standards, and no consequences, it was 
considered to be weak. In the middle were schools with some combination of information, 
standards, and consequences (regardless of source), but not all three. 

We also sought to measure a school’s organizational capacity to allow an examination 
of its association with accountability. For this measure, we employed both the teacher 
survey and codings of school reports. Items were selected that demonstrated the extent to 
which key aspects of organizational capacity - knowledge and skills, the exercise of power 
and authority, and shared commitment and collaborative activity to achieve a clear purpose 
for student learning - were present at a school. The appendix (part II) presents items from 
teacher surveys and school codings used in this analysis.'^ 

In the next section, we discuss results, organized around five key findings. 

Findings on Accountability and Organizational Capacity 

1. Only seven of the 20 schools had strong accountability systems. 

These seven schools had all four components of a complete accountability system: 
information, standards, consequences, and an agent to judge organizational performance. 

Most schools (13 of 20) fell short on explicit standards for student performance and 
significant incentives or consequences for student success. Of these, five had weak systems 
and eight were mid-range. Table 1 presents the schools by level according to the strength of 
their accountability system. 
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INSERT TABLE 1 HERE 



As Table 1 indicates, of the seven strong systems, three were primarily school-based 
(i.e., developed internally), three were mandated by external agents, and one had both 
external and internal components. We also note that four of the seven schools with strong 
systems were high schools, and only one of the seven high schools had a weak system. 

Two schools illustrate differences in the strength of accountability systems. In a weak 
environment, teachers at Fremont High gave state mandated achievement and basic skills 
tests and district criterion tests in the academic subjects. But no one at the school, district or 
state seemed to do much with results. Scores were not published and there were no formal 
consequences for either the school or individual teachers tied to results. Most staff reported 
that they felt little or no pressure for student success on the tests. But many anticipated a 
major change in the following year with the implementation of new state-mandated 
performance assessments with explicit standards. 

In contrast. South Glen High was required by the state to administer end-of-year tests 
in required courses in English, math, the sciences, and social studies. These tests — viewed 
by many staff at the school as tests of factual recall — were intended to hold schools 
accountable for teaching the state curriculum frameworks. The district also required final 
exams for required courses and that scores count 20% of a student’s grade. The state also 
required students to pass competency tests in reading, writing, and grammar to graduate. As 
part of the recent shift to hold individual schools more accountable. South Glen had to 
develop a school improvement plan with 10 indicators of performance other than test results. 
Although standards for adequate performance on plans were not defined by the state, there 
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was the potential threat of closing schools for consistently poor performance. Teachers also 
received up to a 3% salary supplement for meeting individual and school-wide goals. 

South Glen illustrates how external agents, particularly the state, can require elements 
of a strong accountability system. In contrast, Okanagon Middle represents a school for 
which requirements for accountability from the district or state were quite weak. But as 
charter school, Okanagon was charged with creating its own accountability system. Students 
had to meet six "challenges" each year they attended the school and thereby earned the 
Okanagon Certificate. Staff established benchmarks for satisfactory completion of each 
challenge and set the goal that 80% of the graduating eighth graders would earn the 
certificate. In addition, staff created and administered quarterly school-wide performance 
assessments in math and writing. Teachers serving on the school’s curriculum committee 
developed scoring rubrics and significant staff development time was devoted to training the 
whole staff in scoring. Members of the curriculum committee regarded these assessment 
tasks as powerful mechanisms for changing curriculum and instruction, and a way to hold 
each teaching team accountable for the academic progress of their students. 

2. Schools varied considerably in the extent of organizational capacity. 

As we argued above, a school accountability system alone is unlikely to advance 
student learning. It must be combined with a high level of organizational capacity. Table 2 
shows the schools grouped by level and their mean scores of organizational capacity using 
combined, standardized scores from teacher surveys and researchers’ coding. School scores 
are relative to the sample of 20 schools, ranging from a high of 1.64 (Lamar Elementary) to 
a low of -1.81 (Island High) with an overall sample mean of -.01.*^ As a group, the 



elementary schools fared significantly better than the middle and high schools. The overall 
mean for the elementary schools was .78, for the middle schools -.09, and for the high 
schools -.21. The three top schools were all elementary (Lamar, Ashley, and Humboldt) and 
the bottom four were all high schools (South Glen, Fremont, Wallingford, and Island). 

INSERT TABLE 2 HERE 

Lamar Elementary exemplifies a school with high organizational capacity. There, 
teachers team-taught in multi-age clusters (two teachers and 64 students per cluster), planning 
lessons and units together and providing feedback on each other’s teaching. Collaboration 
was focused on implementing the inquiry-based instructional philosophy that was shared by 
the whole staff. Faculty decisions on school-wide curricular themes, hiring, and on-going 
operations of the school provided opportunities for the staff to enhance their program and 
revealed their influence and authority on important issues. 

In contrast, Sumpter Elementary, scoring relatively low for elementary schools and in 
the middle for the whole sample, was a teacher-run school and valued initiative and 
innovation among the staff. But these efforts were largely independent enterprises by 
teachers, resulting in a proliferation of fragmented programs which undermined a common 
purpose and collaborative activity. Sumpter and Lamar represented the contrasting poles of 
an individual vs collective approach to school change. 

3. School accountability and organizational capacity are not necessarily related. 

When we made the distinction between accountability and organizational capacity 
earlier in this paper, we assumed a certain degree of independence between the two. One 
could imagine a school with strong accountability and low organizational capacity and a 



school with high organizational capacity and weak accountability. Having rated the schools 
on each variable, we now examine their actual relationship. 

The association of a school’s organizational capacity to the strength of its 
accountability system is shown in Figure 1. There seems to be no clear relationship. The 
seven schools in the strongest accountability environments ranged from the lowest to near the 
highest in terms of their capacity. A similar trend is evident in the schools with moderate 
and weak accountability systems. These findings are consistent with our analytic point 
above. From a practical standpoint, the challenge is to craft a combination of accountability 

c 

and organizational capacity to improve school performance. 

INSERT FIGURE 1 HERE 

Enhanced school accountability will have positive effects on the organization, or so 
the theory goes. By looking closer at the individual schools with strong accountability 
systems, we shall see the ways in which accountability can influence organizational capacity. 
In our discussion of the final two findings, we highlight these seven schools and demonstrate 
how accountability can, in some contexts, advance organizational capacity while in others, it 
can detract from it. Our discussion clarifies the importance of an internally generated 
system, as well as the interaction of externally-required and internally-generated aspects of 
accountability. 

4. Schools with strong external accountability tended to be low in organizational 
capacity. 

Seven schools had strong accountability systems. Figure 2 shows how each of these 
schools fared in terms of their organizational capacity. As indicated, three schools had 
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strong internally-generated accountability systems, three schools had strong accountability 
systems that were mandated by external agents, and one had a strong system deriving from 
both sources. In Island and South Glen High Schools (scoring -1.81 and -.84 respectively), 
we see how external accountability systems erected important barriers for organizational 
capacity. 

Insert Figure 2 here 

South Glen. South Glen High’s accountability system was described above. The 
relatively high-stakes assessment system required by the state, with its emphasis on basic 
skills and content mastery, was not considered by staff to provide very valid measures of 
student performance. One social studies teacher, for example, complained about questions 
relating to obscure court cases that had appeared on an end-of-course exam. The staff did 
establish specific school-wide goals on attendance, drop-out rates, and the like, but they had 
been unable to agree on a focus for student learning to replace the kind of mastery required 
by the tests. For the most part, teachers remained isolated in their professional work and 
concerned with meeting state mandates in each subject. While teachers gained formal 
positions in the decision-making structures of the school, their influence and involvement 
continued to be limited by state and district directives and a legacy of top-down management. 

Island. Island High represented a similar scenario. Its state had mandated a strong 
accountability system that included subject matter tests with both traditional and open-ended 
questions, portfolios, and student performance events. The state set improvement goals for 
schools in all academic areas. No consequences for failure to meet goals had been 



implemented, but the possibility of eventual school closing by state authorities existed. Staff 
at each school were responsible for developing a plan to meet the goals set out by the state. 

Teacher disagreement in their reactions to the new system weakened organizational 
capacity at Island. Many saw it positively and consistent with their principles for 
restructuring, drawn from the Coalition of Essential Schools (see Sizer, 1985; Muncey & 
McQuillan, 1993). Others disapproved and resisted, claiming that the open-endedness of 
state assessment tasks was not well-suited to Island’s increasingly diverse and disadvantaged 
students. Initiatives such as professional development activities for the new assessments, 
interdisciplinary teaching teams, and faculty committees for teacher decision making, could 
not persuade resisters to join the reform efforts. The state’s strong external accountability 
measures became a bone of contention for staff, failing to stimulate a collective focus on the 
newly established goals for achievement or enhance teachers’ competencies to meet them. 

Careen. The third school with a strong external accountability system. Careen 
Elementary, fared significantly better on the measures of organizational capacity (.58). But 
the organizational capacity they attained was threatened by the external accountability system. 

At Careen, the state required testing in reading and math of all students in grades 3 
through 8. Students in grades 4 and 8 also took exams in writing, science, and social 
studies. Additional information on attendance and drop-out rates was also submitted to the 
state. School standards were set at four levels: exemplary, recognized, acceptable, and low- 
performing. Scores were published by level - district, school, and classroom, which made it 
possible to identify student scores with individual teachers. The state superintendent had the 
authority to audit school districts based on this system and ultimately close low-performing 



schools. A district administrator who was active in starting Careen in the fall of 1992 called 
this "a very high-stakes testing program" which functioned as the state curriculum. 

This system, with its degree of emphasis on basic skills, conflicted directly with the 
program philosophy of Careen Elementary. Curriculum and instruction were focused on 
inquiry-oriented activities that involved real-world application of knowledge. This emphasis 
on "applied learning," as it was called at the school, made the school an appropriate site to 
experiment with alternative forms of assessment. So alongside the state-mandated system, 
students produced portfolios and teachers were involved in serious discussions to establish 
standards of quality for assessing them. Teachers were united in their enthusiasm for applied 
learning and were hired because of their affiliation with this philosophy. They also 
participated, as a school staff and with teachers from other district schools, in professional 

I 

development activities focused on applied learning. According to teachers, the environment 
allowed them to grow and develop their skills as educators. These factors contributed to a 
fairly high degree of organizational capacity. 

On the other hand, teachers were limited in their opportunities to work together 
collaboratively. They taught in self-contained classrooms and continued control from the 
district office limited their influence in further developing and implementing their mission. 
Teachers felt the district failed to follow through on commitments to have the state 
accountability measures waived for the school. There was considerable concern at the school 
regarding these measures, not because of any danger of being closed, but because as an 
experimental school, staff felt the need for legitimacy in the eyes of the community. The 
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strength of the state system pressured teachers to shift their attentions to more traditional 
goals. 

Finally, we turn to the one school that had both a strong external a strong internally- 
developed accountability system. Flinders High scored -.11 on organizational capacity, third 
highest among the seven high schools in the sample. 

Flinders. In addition to state assessment exams, the school was required to submit 
periodic reports to the district that documented its efforts to improve student performance. 
Specifically, Flinders had to submit to the district an annual Academic Improvement Plan 
(AIP) that set student performance goals and outlined strategies for achieving them. Each 
department developed specific goals. Flinders’ staff created a Monitoring and Evaluation 
Committee to ensure that the school achieved its performance standards and other curricular 
goals. The committee issued annual reports that documented each department’s record in 
achieving the goals established in the AIP. These reports, the School Performance Profile 
(SPP), were submitted to school administrators and district officials. The district officially 
recognized schools that achieved their AIP by placing them in a "distinguished school 
program." Schools that fell short of their AIP goals and under-performed on state 
assessments over a period of time became "target schools." It was not clear what kinds of 
consequences resulted from these designations. 

The strong external accountability system contributed to Flinders’ focus on 
conventional standards of student performance. But this failed to boost organizational 
capacity. Flinders’ teachers disagreed on goals for student achievement and many were 
critical of the direction the school was taking in response to the accountability requirements. 



O 



While Flinders had adopted many structural innovations — house programs (schools within 
the school), divisions (interdepartmental organizational units), and teaching teams — this 
compartmentalized organizational pattern made it difficult for faculty to build a strong sense 
of shared purpose. 

In theory, teachers at Flinders had been empowered by site-based management to 
participate in school decision making. The district’s role of holding the school accountable 
for goals the school set for itself appeared on the surface to shift considerable power to the 
school site. Conceivably, this could have opened up opportunities for the development of 
more focused commitments. In practice, governance was conducted by a small cadre of 
people who were the "movers and shakers" within the governance system. Many important ' 
decisions about staffing, curriculum, and budget were made unilaterally by the school’s 
Administrative Team or by the principal. As one teacher noted, "It is still very top down 
here." Frustration with constraints on participatory decision making had led several teachers 
to leave the school. 

5. Strong internal accountability advanced organizational capacity in schools. 

In three schools, strong accountability was generated internally by the school 
community. At Cibola High, Okanagon Middle, and Humboldt Elementary, this stimulated 
consensus on a clear purpose for student learning and staff collaboration to achieve it. As 
shown in Figure 2, these schools had the highest levels of organizational capacity among the 
seven with strong accountability systems. For each school, we will describe how 
accountability seemed to enhance organizational capacity, as well as how capacity contributed 
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to strong internal accountability. The role of external agents in supporting both internal 
accountability and organizational capacity is also discussed. 

Cibola. Cibola demonstrated a relatively high level of organizational capacity (.66), 
especially, for a secondary school. Since its inception in 1985, the school embraced the 
principles of the Coalition of Essential Schools and sought to encourage "students to use their 
minds well, and prepare them to live productive, socially useful and personally satisfying 
lives." 

To support their mission, Cibola created an intricate accountability system that 
combined significant autonomy from regulations with an internally-developed process for on- 
going review and improvement. As part of the alternative high school division of the 
district, the school was insulated from many district mandates and requirements that applied 
to regular schools. The staff did administer the district’s yearly standardized reading test for 
students in grades 7-10 and the state competency tests for graduation. These provided a 
source of credibility for the school, but there were no rewards or sanctions from external 
groups for student performance levels on these tests. 

However, Cibola’s independence from external requirements did not lead to a weak 
environment for accountability. Staff acted on that independence to develop their own 
accountability system that centered on graduation by exhibition and yearly review 
committees. 

To graduate, students completed portfolios in 14 areas, of which 7 were to be 
presented as exhibitions to a graduation committee comprised of the student’s faculty advisor, 
another teacher, another adult of the student’s choosing, and an assigned student. Students’ 



performance on their exhibitions was judged on five criteria: view point, connections, 
evidence, voice, and conventions. There had been continual debate, and some conflict, 
among faculty members about how to apply these standards. But the process helped staff to 
stay focused on standards. One of the co-directors put it this way, "One of the ways that the 
teachers talk about standards over and over again is by sitting together and looking at the 
work of students. Not only in terms of judging it and grading it but also bringing it back to 
the classroom, bringing it back to habits of mind." 

At the end of each year, Cibola invited a committee of "critical friends" to review 
their graduation requirements and portfolio exhibition process. The make-up of these groups 
varied depending on its purpose. For example, a cross-section of assessment experts, 
professional educators, and community members reviewed the school’s entire graduation by 
exhibition process. Since teachers participated in the selection of the external committees 
and the design of this review process, faculty members respected the committees’ 
conclusions, and discussed their implications for curriculum and assessment at the school. 

As one co-director summarized, "We use the outside world as a check on ourselves, but we 
answer to one another and to students and families." 

Cibola’s home-grown accountability system supported all components of school 
organizational capacity. The school was founded with the primary mission to teach students 
to use their minds well, and both the graduation process and evaluative criteria reinforced the 
shared purpose of developing students’ habits of mind. Staff engaged in constructive debates 
about interpreting goals and developing effective strategies for achieving them. This inquiry 
stimulated staff to seek new knowledge and skills. 
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Staff exercised considerable influence over a broad range of issues, from hiring staff 
and developing new curriculum to making programmatic and budgetary decisions. For 
example, by setting standards for evaluating portfolios and exhibitions, teachers enhanced 
organizational capacity by further clarifying their collective focus on the intellectual quality 
of student learning. One of the co-directors argued that this was the most promising 
approach. He said, "I defy anyone anywhere to come in and say the state could do a better 
job of setting standards than Cibola can. I don’t believe it... What people are doing are 
setting national standards in a vacuum; they are... better than nothing, but it’s certainly not 
better than the enacted standard-setting practices that we have here." 

Okanagon. Okanagon had the highest level of organizational capacity (1.17) among 
the middle schools in the study. Like Cibola, external requirements for accountability were 
quite weak, but school staff crafted a strong internal accountability system to reinforce their 
mission. We summarized above major elements of this system. Here we offer more detail 
and explain how the accountability system supported organizational capacity. 

Teachers shared student work, both on the school-wide assessments and on class work 
entered into portfolios, with other restructuring schools in the region. Sponsored by the 
state, this public display created the opportunity for constructive appraisal and feedback, 
thereby contributing to the staff’s understanding of elements of their assessment system. 
According to one teacher, this activity raised "the consciousness of accountability at the 
school." 

The staff at Okanagon also held themselves accountable through an on-going 
longitudinal study, funded by a grant to the school. The school had the district conduct the 
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study which tracked the success of Okanagon graduates in their respective high schools. A 
central feature of the school’s mission was to provide all students with one advanced 
curriculum so they would not be placed in lower academic tracks in high school. Data from 
the first class of graduates indicated that 80% of Okanagon students took an algebra or more 
advanced math course in 9th grade, compared to 50% of students from other middle schools, 
and that grades were the same for both groups. 

Thus, with limited pressure or requirements from external groups, the staff at 
Okanagon, by developing their own system of accountability, enhanced the school’s 
organizational capacity. Teachers exercised significant power and influence in crafting the 
six challenges, standards for meeting them, and specific rubrics for school-wide assessments. 
Staff reinforced their knowledge. and skills through assistance and feedback from external 
experts on alternative assessments and other schools. Assessment tasks and the monitoring 
of their graduates supported their shared concern for intellectual quality and student success 
in high school. 

Humboldt. Humboldt Elementary, one of the schools in the Accelerated Schools 
Project (see Hopfenberg, et al, 1993), showed one of the highest levels of organizational 
capacity (1.50). Here, external requirements for accountability were combined with rigorous 
internally-developed components. 

Like Flinders, Humboldt was required to administer state assessment tests, all 
standardized achievement tests. In accordance with the Accelerated Schools model, 
Humboldt’s staff attempted to ensure that all children scored at grade level according to 
national norms by 5th grade. The entire faculty had embraced this goal. In recent years. 



Humboldt had performed well on these external assessments. Its student achievement scores 
were the highest in its district for elementary schools. School test scores were made public 
by the district, but there were no formal consequences for performance. 

The test scores were also used by faculty members and administrators to informally 
evaluate the effectiveness of individual teachers’ instructional strategies and the 
appropriateness of their curriculum. Teachers who were perceived to be performing less 
than adequately and who failed to respond to the prompts of administrators and other faculty 
members, were not welcomed at the school and ostracized by its professional community. 

Unlike Flinders, conventional academic standards central to the accountability system 
had not hindered the staff at Humboldt from attending to the intellectual quality of teaching 
and learning. Staff incorporated commitments to grade level achievement and more authentic 
learning into their approach to staff development. They used weekly staff meetings as 
workshops for self-training, in which teachers focused heavily on implementing new math 
and language arts curricula. In both subject areas, teachers agreed that the new curricula 
helped them to practice more authentic instruction as reflected in their problem-based 
approaches and applications. They also began work on portfolio assessment. Other topics 
for staff development that year included continued training in the Accelerated Schools model 
and miscue analysis for reading. Clearly, Humboldt teachers did not. perceive a split 
between more traditional and more authentic forms of teaching and learning.*^ 

Humboldt teachers exercised widespread participation and influence to implement 
their instructional goals. Five "cadres" made recommendations to a steering committee, with 
the faculty-as-a-whole serving as the final decision-making body. Grade level teaching 



teams, cross-graded teams, and other formal teacher groups supported broad-based decision 
making by staff. Conversations between teachers in the faculty lounge had produced school- 
wide interdisciplinary curriculum units and innovative instructional programs such as the 
school-within-a-school program. One of the assistant principals indicated that she frequently 
consulted with teachers in the lounge about daily decisions. 

At Humboldt, external accountability requirements reinforced the staff’s concern for 
grade level standards. Their concern for the intellectual quality of student learning, reflected 
in their enthusiasm for new curricula and portfolio assessment, was pursued alongside their 
grade level goals, and both seemed to contribute to an environment in which there were high 
expectations and pressures placed on teachers to perform well. 

Conclusion 

We began by asking how accountability systems might enhance school performance. 
We outlined the four central features of an accountability system (information on 
performance, standards, consequences, and an influential agent) and explained that these 
features raise complex issues for US schools, especially when agents external to schools 
attempt to impose them. We argued further that even when strong external accountability 
can be established, the performance of many schools could languish for lack of school 
organizational capacity. That is, external accountability alone offers no assurance that a 
school faculty will have adequate technical knowledge and skill, sufficient authority to deploy 
resources wisely, or shared commitment to a clear purpose for student learning. 

Theoretically, both accountability and organizational capacity is required for high 
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performance. But it is also possible that strong accountability can be achieved within a 
school community, without prescriptive mandates from a district or state. 

Our study of restructured schools across the United States supported these ideas and 
helped to refine and elaborate them. The design did not allow a formal test of the 
proposition that school performance is highest with both strong accountability and 
organizational capacity.'® But evidence did show that (a) only about a third of the schools 
had strong accountability systems; (b) organizational capacity, which varied significantly 
among the schools, was not related to accountability; (c) schools with strong external 
accountability tended to have low organizational capacity; and (d) strong internal 
accountability tended to reinforce a school’s organizational capacity. 

What are the implications of these findings? We have seen that strong external 
accountability is difficult to implement, and even when it is, it can present serious obstacles 
to or undermine a school’s organizational capacity. We showed that when highly specific 
prescriptive standards connected to high stakes consequences are mandated by external 
authorities, this can deny school staff both the "ownership" (commitment) and the authority it 
needs to work collaboratively to achieve a clear purpose for student learning. 

Does this mean that policy-makers and officials external to schools should abandon 
efforts to strengthen systems of school accountability? No. The findings must be interpreted 
in light of the special sample of schools included in this study. Our conclusions have been 
drawn from the experiences of restructuring schools who had taken "bottom-up" initiative to 
improve. We did not select districts or states with vigorous accountability systems and 
examine their effects on schools within their jurisdictions. The experiences of restructuring 



schools do suggest, however, that efforts of external agencies to strengthen accountability 
should pay increased attention to stimulating the kind of internal accountability that we found 
linked to organizational capacity. As we saw in the examples of Cibola, Okanagon, and 
Humboldt, staff developed explicit school-wide standards focused on student performance, 
mechanisms for collecting and reviewing relevant information, and a culmre of peer pressure 
that served as potentially significant consequences. These seemed to cultivate or reinforce 
widespread consensus around professional norms and offered a focus for collaborative 
activity which stimulated inquiry and searching for additional professional knowledge among 
staff. In each case, however, the school also relied upon important signals and resources in 
the external environment to define its standards and the kind of information collected. 

Districts and states could support internal accountability in several ways. As we saw, 
they can set expectations that individual schools establish their own standards for 
performance and a responsible reporting system. They can support staff development 
opportunities for teachers within a school to formulate performance goals and ways to 
implement them. Districts and states can also establish and reinforce support networks of 
reform-minded schools to assist in sharing standards, assessment techniques, and review 
procedures for evaluation of student learning and school goals. 

The point of such activities is not simply to craft procedures that "let schools do their 
thing." It is to elevate school performance by improving the standards to which schools 
aspire. External agents can make important substantive contributions by offering concrete 
examples of high standards for student performance in specific curriculum areas, approaches 
to assessment that demand high performance, and reliable ways of evaluating student 



performance on the assessments. Without imposing uniform tasks or tests in all schools, 
districts and states, working with school networks and professional organizations, can, in 
these ways, still offer critical leadership in the definition of high standards for student 
achievement. And through such strategies they are more likely to enhance school 
organizational capacity. 

Organizational capacity in schools can probably be built by means other than 
accountability. For example, schools that start anew with an identified mission, that control 
their own hiring of like-minded faculty, and that require parental choice and commitment to 
their educational program have an advantage.’’ But, to the extent that we take arguments 
for increased accountability seriously, strong internal systems of accountability, 
complimented by sponsorship and support from external agencies, can address the challenge 
of organizing technical, human, and social resources into an effective collective enterprise at 
each school. 
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Endnotes 



1 . Others have offered useful critiques of the accountability movement in the United 
States (Levin, 1974; Benveniste, 1985; Darling-Hammond, 1988, 1994; O’Day & 
Smith, 1993; Schrag, 1995; Cohen, in press). Our intent here is not to offer an 
exhaustive or in-depth examination of the movement, but to call attention to issues 
that seemed particularly salient in an empirical study of restructuring schools. 

2. This modifies Rothman’s (1995) definition to acknowledge the potential role of 
constituents other than districts and states. 

3. Other researchers have identified distinct types or approaches to school accountability. 
For example, Kirst (1990), building on Levin (1974), distinguished among the 
following mechanisms to increase accountability: performance reporting, monitoring 
compliance with standards or regulations, incentive systems, market, changing the 
locus of authority on control of schools, and changing professional roles. Darling- 
Hammond (1988) construed accountability as five main types: political, legal, 
bureaucratic, market, and professional. These and other taxonomies illustrate that 
accountability relationships can vary substantially in the standards to be applied, in the 
incentives, and in the constituencies that have power over the provider. 

Our intent here is not to offer a new taxonomy for types, processes, or approaches to 
accountability, but to suggest that at a minimum, a complete accountability system of 
any type would include these four parts. At the same time, we agree with others that 
these minimal attributes, even if implemented, offer no guarantee of universally 
"good” or "desirable" effects. For example. Levin (1974) observes that standards 
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could be low or trivial; Darling-Hammond (1992) argues the need for other conditions 
to maximize high quality learning, equity, and the possibility of school improvement. 
Cohen (in press) shows how little is known about the effects of strong incentives, how 
difficult it would be to implement powerful incentives without more cultural 
agreement on educational goals, and how, in any case, strong incentives are unlikely 
to improve weak schools. 

4. Arriving at standards for school performance may seem most problematic when 
considering standards for all schools in the nation, a state, or a district. We found, 
however, that serious conflict and confusion can ensue even when individual schools 
try to set their own unique standards for student performance (Newmann & 

Associates, 1996). 

5. If the level of reward is small and available to only a small proportion of schools, the 
incentive is less likely to stimulate widescale improvement. Kelly and Odden (1995) 
argue that incentives should be funded sufficiently to reward all schools that meet 
performance targets. 

6. The points are stressed in diverse sources; for example, Kearns & Doyle (1988), 

Clune (1993), O’Day & Smith (1991, 1993), Clune (1993), Committee for Economic 
Development (1994), Consortium on Productivity in the Schools (1995), Darling- 
Hammond (1988), Hanushek (1994), McLaughlin, Shepard & O’Day (1995), 

Rothman (1995). 

7. Literature on restructuring and school reform addresses "capacity building" in a 
general sense (e.g. Darling-Hammond, 1995; McLaughlin, Shepard and O’Day, 
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1995). A more specific conception of organizational capacity is Lawler’s high 
involvement model of organizational productivity (1991, 1992), which he applies 
particularly to organizations that engage in knowledge production, that exist in a 
changing environment, that have complex job tasks requiring constant discretion, and 
that are characterized by task interdependence within the organization. Based largely 
on analysis of business organizations, Lawler argues that performance increases to the 
extent that the following are available to workers at all levels of the organization: 
information about organization’s success and processes; technical knowledge and 
skills; power to control work processes; and rewards for high performance. 

Mohrman, Wohlstetter & Associates (1994) argue that model can be usefully applied 
to school performance. Wohlstetter, Smyer & Mohrman (1994) used the model in a 
comparison between schools that have actively restructured to improve instruction 
versus those struggling to do so. Except for rewards for performance which were 
non-existent in both groups, the actively restructuring schools displayed more 
characteristics of "high involvement" organization. While provocative, these findings 
did not offer a useful test of the Lawler model, because the study failed to include a 
concrete definition of and achievement measures for high performing or high 
productivity schools. 

This definition of school capacity represents a synthesis from research in different 
areas: school restructuring (Newmann & Wehlage, 1995); high involvement 
management in business organizations (Lawler, 1991, 1992) and schools (Wohlstetter, 
Smyer, & Mohrman, 1994); professional community in schools (Louis, Kruse, & 



Associates, 1995; Talbert & McLaughlin, 1994), and other research on school 
organization and reform (e.g., Lee, Bryk, & Smith, 1993; Fullan, 1993; Fine, 1994; 
Lieberman, 1995; Cohen, 1995; O’Day, Goertz, & Floden, 1995) 

9. Other research has documented both the importance and the difficulties of schools’ 
achieving strong professional consensus and collaboration toward clear and demanding 
goals for student learning, especially in high schools. Powell, Farrar & Cohen (1985) 
illustrate the fragmentation of purpose characteristic of American comprehensive 
public high schools. Bryk, Lee & Holland (1993) and Bryk (1996) show how a 
strong religious ideology in Catholic schools, combined with decentralized control and 
voluntary membership fortify unity and cohesion that enhance achievement for all 
students. Talbert & McLaughlin (1994) show that departments, schools and districts 
can support or undermine teacher professionalism and that the quality of 
professionalism depends largely on the strength of teacher professional community 
within subject area departments. 

10. For a more extensive discussion of the criteria for restructuring and school selection, 
see Berends & King (1994) and Newmann (1993a). In four schools, our measure of 
organizational capacity (see note 13) yielded unreliable findings, and these schools 
were omitted from the analysis. 

11. Coding by each researcher was done independently. Any discrepancies between 
researchers in the coding were resolved through discussion until reached consensus. 

12. Typically, researchers rely on either quantitative or qualitative data but our approach 
combined the two. Using both survey data from each school’s staff and our 
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independent coding of school reports enhances confidence in our overall measure of 
organizational capacity. 

13. Both teacher survey items and the coding of school reports were employed to obtain 
measures for the three components of organizational capacity - (1.) knowledge and 
skills, (2.) power and authority, and (3.) shared commitment and collaborative 
activity to achieve a clear purpose. Our measure of the first component assesses the 
effort put into gaining new knowledge and skills. We couldn’t obtain a more direct 
measure of the level or quality of knowledge and skills but think this is an adequate 
approximation. 

For each survey item, responses from teachers at a school were averaged. For each 
school, we then averaged the means for all items that comprised the constructs for 
each component. These school averages for each of the three components were then 
averaged to produce a school mean on organizational capacity, and these means were 
then standardized across schools. Similarly, coding items for each component were 
averaged for each school; these were averaged to produce a school mean, and the 
means were standardized across schools. Standardized scores were then combined for 
an overall mean for each school. The overall sample mean for the standardized 
scores was -.01, with a standard deviation of .90. Measures of organizational 
capacity for each level (elementary, middle, and high) were obtained by averaging the 
mean standardized scores of the six or seven schools at each level. Four schools (one 
elementary, two middle, and one high schools) were dropped from subsequent 
analyses because of large discrepancies in each school’s rankings between the survey 
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and coding measures. Because of the discrepancies, we did not feel we had a reliable 
measure of these schools’ organizational capacity. 

14. These findings on the variation in organizational capacity by level points to a 
potentially important area for further research, which is outside the scope of this 
paper. It would be interesting to compare the different challenges for developing 
organizational capacity, and other key elements of restructuring schools, at the 
different levels. Important work that examines the contexts of US secondary schools 
(e.g., Grossman & Stodolsky, 1995; McLaughlin, Talbert, & Bascia, 1990; Siskin, 
1994) has contributed to understanding the success of reform efforts at that level. 

But, thus far, there has been little comparative analysis of the challenges of building 
capacity at different levels. 

15. Although best seen as a continuum, the distinctiqn between traditional and authentic 
teaching and student achievement reflects the degree to which they meet three criteria: 
construction of knowledge, disciplined inquiry, and value beyond school. For further 
discussion of this distinction and criteria for authentic student performance, see 
Newmann (1993b), Newmann & Associates (1996) and Newmann, Secada, & 
Wehlage (1995). 

16. The study included samples of student written work in mathematics and social studies 
in one grade level at each school. The quality of student performance was scored 
according to standards for authentic achievement presented in Newmann, Secada & 
Wehlage (1995). We compared the average achievement scores between two groups 
of schools: the four rated highest on both accountability (strong internal or external 
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system) and organizational capacity and the four rated lowest on both variables. 
Average achievement of the schools highest in both accountability and organizational 
capacity exceeded average achievement of the schools lowest on both variables by 
about .5 between-school standard deviation. This finding on the connection between 
accountability and capacity to performance is consistent with the conceptual thrust of 
this article, but does not offer adequate evidence of a connection. Due to the small 
sample of schools (24), the comparison did not take into account any contextual 
school factors that have been shown to have major impact on achievement, such as 
socioeconomic status, students’ prior achievement, or school size. 

17. See Newmann & Wehlage (1995) and Wehlage, Osthoff & Porter (1996). 
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Appendix 



This is a partial listing of items for measuring accountability and organizational capacity. A 
complete listing appears a the technical appendix to this paper, available from the authors 
through the Wisconsin Center for Education Research, 1025 W. Johnson St., Madison, WI 
53706. 

I. Accountability System 

Two researchers from each team independently answered the following questions based on 
school reports. Any discrepancies between researchers in the coding were resolved through 
discussion until reached consensus. (This was assessed with 11 items. Four are given as 
examples); 

1 . Does the school participate in a required district testing program? 

3. Are individual teachers judged by standards related to student academic 

f 

performance, other student outcomes, or the teachers’ instructional behavior? 

5. Are teachers subject to any consequences, for meeting or failing to meet 
standards? 
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8. Is the school as a whole, or the principal, subject to any consequences for 
meeting or failing to meet standards? 

II. Organizational Capacity 

A. Two researchers from each team independently answered the following questions 

based on school reports. Any discrepancies between researchers in the coding were 
resolved through discussion until reached consensus. Questions were grouped into 
clusters representing three dimensions of organizational capacity; knowledge and 
skills, power and authority to act, and shared commitment and collaborative activity. 



Knowledge and skills: 

STFDEVl Number of staff development days provided by district for 

school-wide staff development, when all staff participate, school 
year and summer combined (high, medium, low). 

STFDEV3 Amount of staff development that focuses on student learning or 
teaching techniques (high, medium, low). 
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STFDEV4 Professional development is sustained and focused on the 



school’s goals as contrasted to short-term or unrelated to the 
school’s goals (high, medium, low). 

Power and authority (This was assessed with 12 items. Four are given as examples) 
Actual influence (high, medium, low) of teachers over: 



GOV IT 


Curriculum 


GOV2T 


Instruction 


GOV3T 


Student Assessment 


GOV6T 


Staff Development 



Shared commitment and collaborative activity 



ADDTCIl 


Which represents the dominant emphasis in the school? 




A. The staff’s primary concern is the intellectual substance of 
student learning; that is, construction of meaning around 
important ideas (high). 



B. The staff’s primary concern is how to implement or deal 
with procedural/administrative innovations; for example, 
teaming, shared decision making, cooperative learning 
techniques, new scheduling, new grading systems, etc (low). 

C. The staff shows no primary emphasis on either A or B. 

That is, they might show equal concern for each or perhaps a 
different primary emphasis such as offering a trusting 
environment for students (medium). 

PROFCOM2 Collaboration: Teachers work together on restructuring or on 
high quality decisions that affect their daily practice (high, 
medium, low). 

ADDTCI3 The Center recognizes "community" in school as clear shared 

sense of purpose, collaboration, and collective responsibility for 
student learning. Considering all of these three criteria, how do 
you rate the extent of community among staff in the school 
(high, medium, low)? 

B. Organizational capacity was also assessed through several items on the teacher survey. 

Responses across each school were averaged and then the school means from the 
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survey items and the school means from the coding were averaged for a combined 
index of organizational capacity. 

Knowledge and skills: 

TQ22K Teachers in this school are continually learning and seeking new 
ideas (6 pt. scale, strongly disagree [SD] to strongly agree 
[SA]). 

TQ25B Staff development programs in this school permit me to acquire 
important new knowledge and skills (6 pt. scale, SA to SD). 

Power and authority (This was assessed with 14 items. Four are given as examples): 

TQ9B How much control do you feel you have in your target class 

over selecting content, topics, skills to be taught (6 pt. scale, no 
to complete control). 

TQ20F I feel I have influence on the decisions within the school which 
directly affect me (6 pt. scale, SA to SD). 
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TQ21B Teacher influence over school policy: determining the content of 
in-service programs (6 pt. scale, no to great deal of influence). 

TQ21F Teacher influence over school policy: hiring new professional 
personnel (6 pt. scale, no to great deal of influence). 

Shared commitment and collaborative activity (This was assessed with 16 items. Four 
are given as examples): 

TQ22B Most of my colleagues share my beliefs and values about what 
the central mission of the school should be (6 pt. scale, SA to 
SD). 

TQ29 Since the beginning of the current school year, how much time 

per month (on average) have you spent meeting with other 
teachers on lesson planning, curriculum development, guidance 
and counseling, evaluation of programs, or other collaborative 
work related to instruction (6 pt. scale, less than 15 minutes to 
10 or more hours). 
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TQ16D 



My success or failure in teaching students is due primarily to 



factors beyond my control rather than to my own efforts and 
ability (6 pt. scale, SA to SD, rev). 



TQ22A You can count on most staff members to help out anywhere, 
anytime -- even though it may not be part of their official 
assignment (6 pt. scale, SA to SD). 
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Table 1. Schools with Strong, Mid-Range, and Weak Accountability Systems.* 





Strong 


Mid-Range 


Weak 


Elementary 


Careen (E) 
Humboldt (1) 


Ashley 

Lamar 

Sumpter 


Eldorado 
Falls River 


MrooLE 


Okanagon (I) 


Copan 
Red Lake 
S el way 


Morris 

Shining Rock 


High 


Cibola (I) 
Flinders (B) 
Island (E) 
South Glen (E) 


Huron 

Wallingford 


Fremont 



* Strong accountability systems were externally required (E), internally generated (I), or 
both (B). 
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Table 2. Organizational Capacity in 20 Restructuring Schools. 
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Figure 2 

The Relationship of Strong Internal or Extenuil Accountability to Organizational Capacity 
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I. 



Accountability System 



Two researchers from each school research team independently answered the following questions 
based on written research reports on each school. Any discrepancies between researchers in the 
coding were resolved through discussion until they reached consensus. 

1 . Does the school participate in a required district testing program? 

2. Does the school participate in a required state testing program? 

3. Are individual teachers judged by standards related to student academic 
performance, other student outcomes, or the teachers’ instructional behavior? 

4. If yes, are the standards explicit or implicit? 

5. Are teachers subject to any consequences for meeting or failing to meet standards? 

6. If yes, indicate the most significant consequences teachers would face. 

7. Is the school as a whole, or the principal. Judged by any standards related to 
student academic performance, other student outcomes, or teachers’ instructional 
behavior? 

8. Is the school as a whole, or the principal, subject to any consequences for meeting 
or failing to meet standards? 

9. If yes, indicate the most significant consequences the school or principal would 
face. 

10. To what extent have the district and state influenced the accountability system of 
the school (no impact to major impact). 

11. To what extent have the district and state made efforts to influence the 
accountability system of the school (no effort to significant effort). 

If a school had all components of information, standards, and consequences (either externally- 
required, internally-generated, or a combination), that is, positive responses to items 1-9, it was 
considered to have a strong accountability system. At the other extreme, if a school had no 
information beyond standardized testing, no standards, and no consequences, it was considered 
to be weak. In the middle were schools with some combination of information, standards, and 
consequences (regardless of source), but not all three. 

II. Organizational Capacity 

Organizational capacity was measured through a combination of coding items and survey 
responses from teachers. As explained below, standardized scores for coding and for survey 
indicators of organizational capacity were then combined for an overall mean for each school. 

A. Coding Items. Two researchers from each school research team independently answered 
the following questions based on written research reports on each school. Any 
discrepancies between researchers in the coding were resolved through discussion until 
they reached consensus. Questions were grouped into clusters representing three 
dimensions of organizational capacity: knowledge and skills, power and authority to act, 
and shared commitment and collaborative activity. 
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Knowledge and skills: 



STFDEVl 


Number of staff development days provided by district for 


STFDEV3 


school-wide staff development, when all staff participate, 
school year and summer combined (high, medium, low). 

Amount of staff development that focuses on student 
learning or teaching techniques (high, medium, low). 


STFDEV4 


Professional development is sustained and focused on the 
school’s goals as contrasted to short-term or unrelated to the 
school’s goals (high, medium, low). 



Power and authority: 



GOV IT 
GOY2T 
GOV3T 
GOV4T 
GOV5T 
GOV6T 


Actual influence (high, medium, low) of teachers over: 

Curriculum 

Instruction 

Student Assessment 

Budget 

Hiring 

Staff Development 

Actual influence (high, medium, low) of principal over: 


GOV IP 
GOV2P 
GOV3P 
GOV4P 
GOV5P 
GOV6P 


Curriculum 

Instruction 

Student Assessment 

Budget 

Hiring 

Staff Development 



Shared commitment and collaborative activity: 



ADDTCIl 


Which represents the dominant emphasis in the school? 




A. The staff s primary concern is the intellectual substance 
of student learning; that is, construction of meaning around 
important ideas (high). 



B. The staff s primary concern is how to implement or 
deal with procedural/administrative innovations; for 






example, teaming, shared decision making, cooperative 
learning techniques, new scheduling, new grading systems, 
etc (low). 

C. The staff shows no primary emphasis on either A or B. 
That is, they might show equal concern for each or perhaps 
a different primary emphasis such as offering a trusting 
environment for students (medium). 

PROFCOM2 Collaboration: Teachers work together on restructuring or 
on high quality decisions that affect their daily practice 
(high, medium, low). 

ADDTCI3 The Center recognizes "community" in school as clear 
shared sense of purpose, collaboration, and collective 
responsibility for student learning. Considering all of these 
three criteria, how do you rate the extent of community 
among staff in the school (high, medium, low)? 

Coding items for each component were averaged for each school; these were averaged to produce 
a school mean, and the means were standardized across schools. 

B. Survey Items. Organizational capacity was also assessed through the following items on 
a teacher questionnaire. 

Knowledge and skills: 

TQ22K Teachers in this school are continually learning and seeking 
new ideas (6 pt. scale, strongly disagree [SD] to strongly 
' agree [SA]). 

TQ25B Staff development programs in this School permit me to 

acquire important new knowledge and skills (6 pt. scale, SA 
to SD). 



Power and authority: 

TQ9A How much control do you feel you have in your target class 

over selecting textbooks and other instructional materials (6 
pt. scale, no to complete control). 

TQ9B How much control do you feel you have in your target class 

over selecting content, topics, skills to be taught (6 pt. 
scale, no to complete control). 
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TQ9C 


How much control do you feel you have in your target class 
over selecting teaching techniques (6 pt. scale, no to 
complete control). 


TQ9D 


How much control do you feel you have in your target class 
over disciplining students (6 pt. scale, no to complete 
control). 


TQ20B 


Staff are involved in making decisions that affect them (6 
pt. scale, SA to SD). 


TQ20F 


I feel I have influence on the decisions within the school 
which directly affect me (6 pt. scale, SA to SD). 


TQ21A 


Teacher influence over school policy: behavior codes (6 pt. 
scale, no to great deal of influence). 


TQ21B 


Teacher influence over school policy: determining the 
content of in-service programs (6 pt. scale, no to great deal 
of influence). 


TQ21C 


Teacher influence over school policy: setting policy on 
grouping students in class by ability (6 pt, scale, no to great 
deal of influence). 


TQ21D 


Teacher influence over school policy: establishing the 
school curriculum (6 pt. scale, no to great deal of 
influence). 


TQ21E 


Teacher influence over school policy: determining the 
school’s schedule (including teacher prep periods) (6 pt. 
scale, no to great deal of influence). 


TQ21F 


Teacher influence over school policy: hiring new 

professional personnel (6 pt. scale, no to great deal of 
influence). 


TQ21G 


Teacher influence over school policy: planning school 
budgets (6 pt. scale, no to great deal of influence). 


TQ21H 


Teacher influence over school policy: determining specific 
professional and teaching assigrunents (6 pt. scale, no to 
great deal of influence). 
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Shared commitment and collaborative activity: 



TQ22B 


Most of my colleagues share my beliefs and values about 




what the central mission of the school should be (6 pt. 
scale, SA to SD). 


TQ22C 


Goals and priorities for the school are clear (6 pt. scale, SA 
to SD). 


TQ22G 


There is a great deal of cooperative effort among staff 
members (6 pt. scale, SA to SD). 


TQ22H 


I make a conscious effort to coordinate the content of my 
course with other teachers (6 pt. scale, SA to SD). 


TQ27B 


Since the beginning of the current school year, how often 
have you received useful suggestions for curriculum 
materials from colleagues in you department (6 pt. scale, 
never to 10 or more times). 


TQ27C 


Since the beginning of the current school year, how often 
have you received useful suggestions for teaching 
techniques or student activities from colleagues in you 
department (6 pt. scale, never to 1 0 or more times). 


TQ29 


Since the beginning of the current school year, how much 
time per month (on average) have you spent meeting with 
other teachers on lesson plaiming, curriculum development, 
guidance and counseling, evaluation of programs, or other 
collaborative work related to instruction (6 pt. scale, less 
than 15 minutes to 10 or more hours). 


TQIO 


To what extent do you feel that you have been successful 
in providing the kind of education you would like to 
provide for the students in the target class (4 pt. scale, little 
to very successful). 


TQ16B 


Many of the students are not capable of learning the 
material I am supposed to teach (6 pt. scale, SA to SD, 
rev). 


TQ16C 


The attitudes and habits my students bring to my class 
greatly reduce their chances for academic success (6 pt. 
scale, SA to SD, rev). 



TQ16D 


My success or failure in teaching students is due primarily 
to factors beyond my control rather than to my own efforts 
and ability (6 pt. scale, SA to SD, rev). 


TQ16F 


I feel responsible for the students I teach, but not other 
students in the school (6 pt. scale, SA to SD, rev). 


TQ20G 


I sometimes feel it is a waste of time to do my best as a 
teacher (6 pt. scale, SA to SD, rev). 


TQ20H 


I am certain I am making a difference in the lives of my 
students (6 pt. scale, SA to SD). 


TQ22A 


You can count on most staff members to help out 
anywhere, anytime — even though it may not be part of 
their official assignment (6 pt. scale, SA to SD). 


TQ22E 


Teachers are expected to help maintain discipline in the 
entire school, not just their classroom (6 pt. scale, SA to 
SD). 



For each survey item, responses from teachers at a school were averaged. For each school, we 
then averaged the means for all items that comprised the constructs for each component. These 
school averages for each of the three components were then averaged to produce a school mean 
on organizational capacity, and these means were then standardized across schools. 

The final indicator of organizational capacity for a school was the sum of its standardized scores 
based on the coding and the survey items. 
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